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ABSTRACT 

This  paper  describes  the  development,  flight- testing 
and  demonstration  of  technologies  for  the  autonomous 
landing  of  a  Yamaha  RMAX  helicopter  at  non- 
cooperative  sites  without  the  aid  of  GPS.  The  Yamaha 
RMAX  used  for  these  flight  trials  has  been  modified  to 
include  stereo  cameras,  a  laser  range  finder,  and  an 
avionics  payload.  Machine  vision  stereo  range  mapping 
is  used  to  generate  an  accurate  terrain  representation,  and 
a  safe  landing  area  determination  algorithm  is  used  to 
select  the  safest  landing  point  within  the  terrain.  A 
machine  vision  self-localization  system  is  used  to 
measure  the  helicopter  position  during  the  descent  and 
landing  when  GPS  is  unavailable.  The  software  and 
hardware  architecture  of  the  flight  system  is  presented 
and  each  system  component  is  described.  Results  and 
lessons  learned  from  the  flight  evaluation  and 
optimization  of  the  individual  components  are  reported 
as  well  as  the  overall  system  performance  with  respect  to 
a  set  of  objective  metrics  for  the  autonomous  landing 
flight  trails. 

1.  INTRODUCTION 

With  the  intent  of  the  US  armed  forces  to  field  an 
increasing  number  of  Unmanned  Aerial  Vehicles 
(UAVs)  as  part  of  the  Future  Combat  System  (FCS), 
particularly  Rotorcraft  UAVs  (RUAVs)  and  other 
Vertical  Take-Off  and  Fanding  (VTOF)  UAVs,  there  is  a 
clear  need  for  an  autonomous  landing  capability  to 
remote,  unprepared  (and  possibly  cluttered)  sites.  Such  a 
capability  would  also  need  to  include  a  self-localization 
component  since  GPS  signals  may  be  intermittent  or 
unavailable  during  the  landing  task  due  to  electronic 
jamming  or  occlusion  of  GPS  satellites.  In  addition  to 
the  difficulty  of  finding  a  suitable  landing  point  and 
navigating  to  this  landing  point  without  GPS,  all  terrain 
sensing,  information  processing  and  decision-making 
functions  must  be  performed  on-board  without  the  need 
for  operator  interaction.  Typical  operational  scenarios 
that  would  benefit  from  such  a  capability  include:  perch- 
and- stare  surveillance  operations;  ground  loiter  to 
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conserve  fuel  and  other  system  resources;  precision 
supply  delivery  and  Forward  Arming  and  Refueling 
Point  (FARP)  operations;  remote  recovery  of  the  RUAV; 
and  forced  landing  contingencies  resulting  from  on¬ 
board  failures  or  loss  of  communication. 

Current  technologies  for  the  landing  of  UAVs  are 
mostly  limited  to  using  an  external  pilot,  recovery  net,  or 
auto-land  system  requiring  specially  prepared  and 
instrumented  landing  sites.  These  technologies  often 
preclude  UAVs  from  landing  autonomously  or  from 
landing  in  un-prepared  environments  where  the  terrain 
profile  is  uncertain  and  possibly  cluttered. 

The  Precision  Autonomous  Fanding  Adaptive 
Control  Experiment  (PAFACE)  Army  Technology 
Objective  (ATO)  was  formulated  to  address  some  of 
these  current  limitations  by  developing,  integrating  and 
demonstrating  technologies  for  reliable,  autonomous 
landings  of  RUAVs.  The  PAFACE  program  was  a 
three-year  activity  (completed  in  FY06)  performed  by 
the  US  Army  Aerofhghtdynamics  Directorate  (AFDD)  at 
Moffett  Field  in  California  and  concluded  with  a  public 
demonstration  of  a  Yamaha  RMAX  helicopter  making  a 
number  of  successful  landings  to  cluttered  landing  sites 
without  the  aid  of  GPS  for  navigation.  Supporting 
efforts  were  provided  by  the  Army/NASA  Autonomous 
Rotorcraft  Program  (ARP),  Brigham  Young  University 
(BYU),  and  the  Mobility  and  Robotics  Group  at  JPF. 

The  technical  approach  developed  to  achieve 
autonomous  landing  capabilities  first  uses  passive  stereo 
ranging  to  build  a  3D  terrain  profile  of  the  potential 
landing  site.  The  stereo  ranging  algorithm  uses  a  pair  of 
images  from  digital  cameras  mounted  on  the  helicopter 
to  build  the  3D  profile  of  the  terrain  visible  in  both 
cameras. 

A  landing  site  selection  algorithm  then  inspects  the 
terrain  map  to  determine  if  the  site  is  suitable  for  landing 
and  to  select  the  point  that  best  meets  a  set  of  vehicle  and 
mission  landing  site  requirements.  The  requirements 
ensure  that  the  chosen  landing  point  has  a  surface  slope 
within  the  physical  capabilities  of  the  helicopter,  is  a 
minimum  distance  from  obstructions,  and  has  a  surface 
roughness  below  a  given  threshold. 
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Navigation  to  the  selected  landing  point  without 
GPS  is  achieved  using  machine-vision  self-localization 
algorithms  that  provide  the  helicopter  position  relative  to 
the  ground  terrain  as  the  helicopter  descends  to  the 
chosen  landing  point.  Stable  hover  and  descent  is 
achieved  using  this  system  over  various  surface  textures 
without  ground  based  markings  or  instrumentation. 

Initial  work  on  the  PALACE  program  independently 
demonstrated  each  of  the  core  machine  vision 
technologies  (Hintze  et  al.,  2004),  and  validated  their 
utility  in  both  simulation  and  flight.  This  was  followed 
by  the  construction  of  an  integrated  simulation 
environment  (Theodore  et  al.,  2005)  for  the  development 
of  the  landing  procedures  and  integration  of  the  machine 
vision  algorithms,  vehicle  dynamics,  and  control  laws. 
The  performance  of  the  individual  machine  vision 
algorithms  was  evaluated  using  the  simulation.  The 
simulation  also  provided  a  level  of  risk  reduction  when 
transitioning  the  PALACE  landing  technologies  to  flight 
trials.  Finally,  flight  development,  testing  and  validation 
of  the  autonomous  landing  technologies  were  performed 
on  a  Yamaha  RMAX  RUAV  (Theodore  et  al.,  2006). 
The  in-flight  performance  of  the  individual  technologies 
was  evaluated,  as  well  as  the  overall  performance  of  the 
landing  system.  Repeat  successful  landings  were 
performed  to  a  number  of  different  surfaces,  with 
different  obstacle  fields,  and  with  variable  wind 
conditions. 

Table  1  lists  the  quantitative  metrics  and  target 
performance  objectives  for  the  PALACE  program.  The 
first  three  metrics  are  constraint  values  for  the  landing 
site  selection  algorithm  and,  for  this  case,  are  a  function 
of  the  geometry  of  the  Yamaha  RMAX  helicopter.  The 
following  metric  specifies  the  landing  accuracy  and 
accounts  for  drift  in  vehicle  position  during  the  vision- 
based  descent.  The  next  requirement  specifies  that  the 
self- localization  must  run  with  a  processing  time  of  less 
than  100  msec  in  order  produce  a  position  estimate  at 
10Hz.  The  final  two  objectives  specify  that  the  landing 
site  selection  algorithm  should  choose  a  safe  landing  site 
in  under  5  seconds  with  a  greater  than  98%  success  rate. 

This  paper  first  gives  an  overview  of  the  fight 
software  and  hardware  architecture  and  describes  each  of 
the  individual  components.  This  includes  the  on-board 
mission  manager  element  that  coordinates  each 
component  of  the  system  and  provides  all  of  the 
intelligence  and  decision-making  capabilities  to  enable 
autonomous  landings.  The  next  section  presents  results 
from  the  flight  trials  on  the  Yamaha  RMAX,  including 
stereo  range  mapping,  landing  site  selection,  and  self¬ 
localization  without  GPS.  The  final  section  presents 
concluding  remarks  and  discusses  the  ability  of  the 


Table  1.  PALACE  program  performance  metrics  and 
objective  values  for  the  Yamaha  RMAX  vehicle. 


Quantitative  Metric 

Project 

Objective 

Landing  Site  Size 

<  7.0  m 

Landing  Surface  Slope 

<  15  deg 

Landing  Surface  Roughness 

<  10  cm 

Landing  Accuracy 

<  1 .25  m 

Feature-Tracking  Cycle  Time 

< 100  msec 

SLAD  Calculation  Time 

<  5  sec 

SLAD  Success  Rate 

>  98% 

PALACE  system  to  meet  the  set  of  objectives  listed  in 
Table  1. 

2.  PALACE  SYSTEM  ARCHITECTURE 

Figure  1  shows  a  schematic  of  the  hardware  and 
software  architecture  used  for  the  PALACE  flight  trials. 
Each  element  of  this  architecture  is  described  in  the 
following  sections. 

2.1  ARP  RMAX  hardware  and  sensors 

The  flight  trials  and  demonstrations  for  the 
PALACE  program  were  performed  using  a  modified 
Yamaha  RMAX  that  is  part  of  the  Army/NASA 
Autonomous  Rotorcraft  Project  (ARP)  (Whalley  et  al., 
2003).  The  Yamaha  RMAX  is  a  small-scale  helicopter 
with  a  rotor  diameter  of  3.12  m  and  an  empty  mass  of  66 
kg.  The  maximum  payload  is  28  kg. 

The  RMAX  operated  by  ARP  (Figure  2)  has  been 
modified  to  include  an  avionics  payload  and  various 
sensors.  The  avionics  payload  includes:  a  navigation  and 
flight  control  computer,  an  experimentation  computer,  an 
IMU,  a  GPS  receiver,  and  communications  equipment. 
The  experimentation  computer  is  a  compact-PCI  with  a 
Pentium  III  running  at  700Mhz  and  hosts  the  PALACE 
software  and  machine- vision  algorithms.  A  pair  of 
monochrome  640x480  resolution  stereo  cameras  are 
mounted  on  either  end  of  a  vibration- isolated  articulated 
stub  wing  with  a  stereo  baseline  of  1.1  m.  Figure  3 
shows  the  camera  arrangement  on  the  left  end  of  the  stub 
wing  with  monochrome  and  color  cameras  (the  PALACE 
system  uses  only  the  monochrome  camera  for  stereo 
ranging).  Finally  a  SICK  scanning  laser  is  mounted 
under  the  nose.  For  the  PALACE  flight  tests,  only  the 
laser  return  to  the  ground  at  the  center  of  the  camera 
image  is  used  to  simulate  a  simple  laser  range  finder. 

2.2  ARP  flight  control  laws  (CLAW) 

The  CLAW  block  of  the  flight  architecture  (see 
Figure  1)  contains  the  inner-loop  and  outer-loop  control 
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Figure  1.  PALACE  flight  hardware  and  software  architecture. 


Figure  2.  ARP  RMAX  research  aircraft. 


laws,  as  well  as  the  mode  switching  elements.  The 
inner-loop  provides  the  primary  attitude  stabilization  and 
attitude  control  of  the  helicopter.  The  outer-loop 
includes  a  waypoint  navigation  controller,  which  is  made 
up  of  a  path  smoother  and  a  path  follower.  When  a  list 
of  waypoints  are  received  into  the  CLAW  block  (from 
the  operator  or  from  the  PALACE  mission  manager) 
they  are  first  passed  to  the  path  smoother,  which 
produces  a  larger  list  of  waypoints  that  describes  a 
smooth  path  between  the  original  waypoints.  The  path 


Figure  3.  ARP  RMAX  avionics  payload  and  stub  wing 
with  left  cameras. 


follower  then  takes  the  desired  smooth  path  and  provides 
a  steady  stream  of  commands  to  the  inner-loop  to  guide 
the  helicopter  along  the  path. 

The  CLAW  block  provides  a  number  of  different 
modes  for  localization  during  waypoint  navigation.  The 
default  is  to  use  the  signal  returned  by  the  on-board  GPS 
receiver  for  navigation.  A  second  mode  allows  an 
external  position  to  be  input  into  the  CLAW  block, 
which  is  then  used  for  navigation.  In  the  PALACE 
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Figure  4.  Flowchart  ofMPE  processing. 


Figure  5.  Stereo  images  and  resulting  range  map. 


architecture,  this  external  position  is  the  ‘pseudo-GPS’ 
signal  from  the  machine-vision  position-estimation 
system.  A  third  mode  uses  inertial  navigation  to  provide 
a  position  estimate  based  on  the  attitude  and  acceleration 
measurements.  Each  of  these  navigation  modes  is 
selectable  by  sending  a  request  to  the  CLAW  block, 
which  produces  a  smooth,  transient-free  transition  to  the 
selected  mode. 

2.3  Monocular  position  estimation  (MPE)  system 

The  MPE  system  provides  a  helicopter  position  that 
is  independent  of  GPS  and  is  used  for  navigation  during 
the  landing  when  GPS  is  assumed  to  be  unavailable.  The 
MPE  system  takes  the  left  camera  images,  the  laser  range 
to  the  center  of  the  camera  image,  and  information  about 
the  pose  of  the  camera  to  measure  the  position  of  the 
helicopter  relative  to  a  fixed  point  on  the  ground  that  is 
being  tracked  over  time. 

Figure  4  shows  a  schematic  of  sensor  data 
processing  to  produce  the  position  estimate.  The 
machine- vision  feature- tracking  algorithm  was 
developed  by  JPL  (Johnson  et  al.,  2001)  and  is  able  to 
lock  onto  surface  features  and  accurately  track  them 
from  frame  to  frame.  The  MPE  system  runs  in  real-time 
at  10  Hz  and  produces  a  pseudo-GPS  position  estimate. 
The  rate  and  format  of  the  pseudo-GPS  signal  is  the 


same  as  the  signal  from  the  GPS  receiver,  which  required 
no  changes  to  the  control  system  to  integrate  the 
machine-vision  position  estimate.  Additional  details  of 
the  MPE  system  can  be  found  in  Ref.  (Theodore  et  al., 
2006). 

2.4  Landing  site  selection  system 

The  landing  site  selection  system  used  for  the 
PALACE  program  consists  of  two  steps.  The  first  step 
involves  creating  a  stereo  range  map  to  represent  the 
surface  terrain  profile.  The  stereo  ranging  algorithm 
utilizes  images  from  a  pair  of  stereo  cameras  along  with 
pose  information  for  the  cameras.  The  stereo  range  map 
is  created  by  determining  pixel  disparities  between 
features  in  the  left  and  right  camera  images  (Figure  5). 

The  second  step  in  the  landing  site  selection  process 
utilizes  a  Safe  Landing  Area  Determination  (SLAD) 
algorithm.  This  algorithm  applies  a  set  of  landing  point 
constraints  to  the  range  map  to  find  all  safe  landing 
regions,  and  then  to  choose  the  optimum  landing  point. 
Three  constraints  on  the  landing  point  are  applied  to 
ensure  the  surface  slope  is  below  a  given  tolerance,  any 
surface  obstructions  or  roughness  are  below  a  given  size, 
and  the  range  to  the  nearest  obstacle  is  greater  than  a 
given  distance.  For  the  ARP  RMAX  the  slope  limit  is 
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Figure  6.  SLAD  operator  interface  showing  results  of 
landing  point  selection. 

set  to  a  maximum  of  15  degrees,  and  the  safe  distance 
constraint  (open-area  diameter)  is  a  minimum  of  7.0  m. 


GPS-based  Navigation 

30  Meters 

-  GPS  navigation  to  landing  zone 

-  Start  landing  procedure  at  30  meters  AGL 

-  Run  SLAD  to  determine  safe  landing  point 

-  Begin  tracking  landing  point 

-  Move  to  bring  landing  point  to  center  of  image 

-  Descend  to  24m  AGL  using  GPS  for  navigation 

-  Re-run  SLAD  to  refine  landing  point 

-  Begin  tracking  updated  landing  point 

-  Descend  to  18m  AGL  using  GPS  for  navigation 

-  Re-run  SLAD  to  refine  landing  point 

-  Begin  tracking  updated  landing  point 


18  Meters 


24  Meters 


Machine  Vision  based  Navigation 

12  Meters 

-  Descend  to  12m  AGL  using  GPS  for  navigation 

-  Switch  to  MPE,  pseudo-GPS  for  positioning 

-  Re-run  SLAD  to  refine  landing  point 

-  Begin  tracking  updated  landing  point 

6  Meters 

-  Descend  to  6m  AGL  using  MPE  for  navigation 

-  Verify  landing  point  by  checking  roughness 

Inertial  Navigation 

2  Meters 

-  Descent  to  2m  AGL  using  MPE  for  navigation 

-  Initiate  final  landing  sequences 

-  Switch  to  Inertial  Navigation  for  positioning 

Ground 

-  Descend  to  ground  using  Inertial  Navigation 

-  Weight  on  wheels  switches  for  vehicle  shutdown 

The  roughness  constraint  defines  the  maximum 
allowable  size  of  ground  features  or  obstructions  for 
landing.  Any  terrain  roughness  or  surface  objects 
physically  larger  than  the  roughness  constraint  value  will 
be  recognized  as  obstacles  by  the  SLAD  algorithm  and 
the  surrounding  terrain  will  be  rejected  as  a  safe  landing 
area.  Since  the  resolution  of  stereo  ranging  varies  with 
the  physical  distance  above  the  ground,  the  value  of  the 
roughness  constraint  must  be  set  dynamically  based  on 
the  height  above  ground  level  (AGL).  For  example, 
from  6  m  AGL,  the  system  is  capable  of  rejecting 
landing  sites  with  obstacles  as  small  as  10  cm,  while 
from  30  m  AGL  it  can  reject  sites  with  obstacles  only  as 
small  as  1.2  m.  Therefore,  the  landing  point  selection 
system  has  to  be  run  at  different  altitudes  during  the 
descent  with  successively  tighter  roughness  constraint 
values. 

Figure  6  shows  an  example  of  the  SLAD  operator 
interface.  The  display  shows  results  from  the  safe 
landing  site  selection  system,  in  this  case  with  two 
obstacles  in  the  camera  field  of  view.  The  yellow  region 
shows  the  portion  of  the  camera  image  where  no  range 
data  are  available  due  to  non- overlapping  left-right 
image  features  or  the  close  proximity  of  the  edge  of  the 
image.  The  points  in  red  violate  one  or  more  of  the 
landing  point  selection  constraints  and  are  determined  to 
be  unsafe  for  landing.  For  this  particular  case,  the  points 
in  the  red  region  are  either  too  close  to  the  obstacles,  or 
too  close  to  the  inner  edge  of  the  yellow  region,  which  is 
treated  as  an  obstacle  since  no  terrain  range  information 
is  available  beyond  this  border.  The  green  region 
indicates  the  points  that  meet  all  of  the  constraints  and 
are  considered  safe  to  land.  Finally,  the  black  *+’ 


Figure  7.  PALACE  landing  sequence  during  the  descent 
from  30m  to  the  ground. 

indicates  the  point  that  best  meets  the  constraints,  and  the 
circle  shows  the  diameter  of  the  safe-distance  constraint. 

The  information  in  Figure  6  is  presented  to  the 
operator  each  time  a  new  landing  point  is  selected  and 
allows  the  operator  to  verify  the  landing  point.  The 
operator  also  has  the  option  of  selecting  an  alternate 
landing  point  if  necessary. 

2.5  PALACE  mission  manager 

The  PALACE  mission  manager  software  is  the  heart 
of  the  system  and  unifies  the  various  elements.  It  also 
provides  the  decision-making  and  coordination 
capabilities  required  to  fly  complete  PALACE  landings. 
Figure  7  shows  the  steps  in  the  PALACE  landing 
procedure  as  the  helicopter  descends  from  30  m  AGL  to 
the  ground. 

The  landing  procedure  starts  with  the  helicopter 
arrival  at  the  landing  zone,  which  has  been  specified  by 
the  operator  as  a  general  location  rather  than  a  specific 
landing  point.  The  cameras  on  the  RMAX  are  set  to  a 
pitch  angle  of  -60  degrees  so  that  the  helicopter  will 
descend  and  land  at  this  angle.  The  descent  angle,  or 
glideslope,  is  set  the  same  as  the  camera  angle  so  that  the 
landing  point  remains  nominally  in  the  center  of  the 
camera  image  as  the  helicopter  descends. 

The  helicopter  moves  under  GPS  waypoint 
navigation  to  view  the  nominal  landing  point  location 
specified  by  the  operator  from  an  altitude  of  30  m  AGL. 
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At  this  point,  the  stereo  ranging  and  SLAD  algorithms 
are  run  to  select  an  initial  landing  point  that  best  meets 
the  set  of  slope,  roughness  and  safe  distance  constraints. 
The  machine  vision  feature-tracker  estimates  the  location 
of  the  landing  point  and  starts  tracking  this  point.  The 
PALACE  mission  manager  then  calculates  and  sends  a 
waypoint  to  move  the  helicopter  laterally  and  vertically 
so  that  the  selected  landing  point  will  be  in  the  center  of 
the  camera  image  from  24m  AGL. 

At  24  m,  the  stereo  ranging  and  SLAD  algorithms 
are  run  again  to  refine  the  landing  point  location  with  a 
tighter  roughness  constraint  to  reject  regions  with  smaller 
obstacles  that  could  not  be  detected  from  the  higher 
altitude.  The  feature-tracker  is  then  re-initialized  with 
the  updated  landing  point  and  a  waypoint  is  set  to  bring 
this  point  to  the  center  of  the  camera  image.  Another 
waypoint  is  set  at  an  altitude  of  18  m  along  the 
glideslope  and  the  vehicle  descends  further.  The  same 
procedure  happens  at  18m  AGL  to  bring  the  helicopter 
down  to  12m  AGL. 

At  12  m,  the  CLAW  block  is  instructed  to  switch 
from  GPS  to  the  pseudo-GPS  position  estimate  returned 
by  the  MPE  system  for  navigation.  The  descent  and 
landing  from  this  point  is  flown  entirely  without  using 
GPS.  The  SLAD  algorithm  is  run  again  to  refine  the 
landing  point  and  the  helicopter  descends  to  6  m  AGL. 

At  6  m  AGL,  the  stereo  ranging  and  SLAD 
algorithms  are  run  for  the  final  time  to  verify  that  the 
final  landing  zone  is  obstacle-free  (within  the  final 
roughness  constraint  value)  and  safe  for  landing.  Once 
the  final  landing  point  has  been  verified,  the  helicopter 
descends  to  2  m  AGL  where  the  command  to  initiate  the 
final  landing  script  is  given.  The  helicopter  begins  the 
final  descent  using  MPE  for  navigation.  When  MPE 
goes  out  of  range  (too  low  for  reliable  tracking),  CLAW 
switches  to  inertial  navigation  for  the  final  portion  of  the 
landing.  The  total  time  while  in  inertial  navigation  is 
generally  less  than  about  10  seconds  so  that  there  is  little 
drift  in  the  absolute  position  during  this  period.  Weight- 
on-wheels  switches  are  triggered  when  the  vehicle 
touches  the  ground  to  complete  the  landing. 

3.  FLIGHT  TEST  RESULTS 

Among  the  most  critical  technologies  for  the 
automated  landing  task  are  the  machine  vision 
algorithms  and  the  integration  of  these  algorithms  with 
the  RMAX  hardware  and  control  laws.  Component 
flight  tests  were  first  performed  to  validate  and  optimize 
the  in-flight  performance  of  the  various  machine  vision 
elements.  Following  this,  flight  trials  of  the  complete 
system  were  performed  to  validate  the  mission  manager 
functionality  and  decision-making  capabilities,  the 


integration  of  the  machine  vision  elements  into  the 
complete  landing  procedure,  and  the  smooth  transition 
between  different  control  modes.  This  ultimately  led  to 
complete  landings  on  various  surfaces  and  obstacle 
fields.  Upwards  of  30  flights  to-date  have  included  test 
points  for  the  development  and  evaluation  of  the 
PALACE  components  and  system  (Theodore  et  al., 
2006).  Upwards  of  40  successful  landings  using  the 
complete  system  have  been  performed  to  date  on  various 
surfaces  and  obstacle  fields. 

3.1  Monocular  position  estimation  (MPE)  evaluations 

The  monocular  feature-tracker  and  MPE  algorithms 
take  left  camera  images,  the  laser  range  to  the  center  of 
the  camera  images,  and  information  about  the  pose  of  the 
camera  to  estimate  the  position  of  the  helicopter  position 
relative  to  a  fixed  point  on  the  ground.  This  allows  the 
MPE  system  to  provide  a  self-localization  capability  that 
is  used  for  navigation  during  the  final  portion  of  the 
descent  from  12  m  AGL  to  the  ground  without  GPS. 

The  performance  of  the  MPE  system  was  evaluated 
in-flight  for  a  number  of  different  conditions,  including 
variations  in  atmospheric  conditions  (wind  speed,  wind 
direction  and  level  of  turbulence),  lightning  conditions 
(overcast  and  full  sun),  surface  textures  (grass,  concrete, 
asphalt  and  gravel),  and  height  above  the  ground.  The 
performance  metrics  that  were  evaluated  during  flight 
were  the  amount  of  tracking  drift,  the  processing  time 
required  for  each  cycle  of  tracking  and  position 
estimation,  and  the  ability  of  the  helicopter  to  hold 
position  and  maintain  the  correct  descent  path  while 
navigating  to  the  ground. 

Flight  tests  of  the  MPE  system  in  separate  hover, 
climb  and  descent  tests  revealed  that  MPE  is  robust  for 
different  surfaces,  with  similar  performance  seen  over 
grass,  concrete,  asphalt  and  gravel.  MPE  is  also  robust 
to  lighting  conditions  (overcast  and  full  sun)  and 
atmospheric  conditions  (wind  speed,  wind  direction  and 
level  of  turbulence)  with  similar  amounts  of  tracking 
drift  seen  in  each  case.  The  total  amount  of  tracking  drift 
observed  in  each  case  was  small  and  limited  to  about  15- 
20  cm  per  minute. 

Figure  8  shows  the  Easting,  Northing  and  height 
time  histories  during  a  climb  from  12  m  AGL  to  30  m 
AGL  using  the  vision-based  self-localization  system. 
The  total  amount  of  drift  for  this  case  after  about  5 
minutes  without  GPS  was  less  than  1  m.  This  amount  of 
drift  is  well  within  the  2  m  margin  included  in  the  SLAD 
safe  distance  constraint  to  account  for  tracking  drift, 
particularly  since  the  amount  of  time  to  navigate  from  12 
m  AGL  to  the  ground  is  only  on  the  order  of  one  to  two 
minutes. 
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Figure  8.  Time  histories  comparing  actual  position 
(GPS)  and  estimated  position  (MPE)  for  a  climb  from 
12m  AGL  to  30m  AGL. 


3.2  Stereo  ranging  evaluations 

The  generation  of  an  accurate  range  map  from  a  pair 
of  stereo  camera  images  is  the  key  element  in  the 
selection  of  a  safe  landing  point.  If  the  range  map  does 
not  represent  the  ground  terrain  and  obstacles  with 
sufficient  accuracy,  then  the  SLAD  algorithm  may  not 
select  the  best,  or  even  a  safe  landing  point,  and  may  also 
reject  a  potentially  safe  landing  point. 

The  key  performance  measure  associated  with  stereo 
ranging  is  the  amount  of  uncertainty  (or  noise)  in  ranging 
flat  ground,  which  gives  an  indication  of  stereo  ranging 
resolution.  The  resolution  places  a  lower  limit  on  the 
size  of  obstacles  that  can  be  resolved  from  the  ranging 
noise  and  is  used  as  the  basis  for  setting  the  roughness 
constraint  values  for  the  SLAD  algorithm. 

Table  2  lists  the  roughness  constraint  values  versus 
height  for  asphalt  and  grass  surfaces.  These  values  are 
derived  from  the  amount  of  noise  in  the  stereo  range 
maps  observed  when  ranging  flat  surfaces,  and  represent 
the  size  of  obstacles  that  are  clearly  distinguishable  from 
the  noise  in  the  range  maps.  These  roughness  constraint 
values  therefore  place  an  lower  limit  on  the  size  of 


Table  2.  Roughness  constraint  values  as  a  function  of 
altitude  for  asphalt  and  grass. 


Roughness  Constraint  (m) 

Height  AGL  (m) 

Asphalt 

Grass 

30 

1.20 

1.50 

24 

0.80 

1.00 

18 

0.40 

0.55 

12 

0.25 

0.30 

6 

0.10 

0.10 

obstacles  that  will  cause  a  surface  to  be  rejected  by  the 
SLAD  algorithm.  Obstacle  sizes  less  than  these  values 
may  or  may  not  be  rejected.  The  values  indicate  that 
from  6  m  AGL  over  asphalt,  the  system  will  reliably 
reject  landing  surfaces  with  obstacles  of  height  10  cm 
and  taller. 

3.3  SLAD  flight  evaluations 

The  SLAD  algorithm  takes  a  range  map  and 
combines  it  with  a  set  of  landing  point  constraints  to  first 
calculate  a  safe  landing  map,  and  second,  choose  the 
optimum  landing  point.  For  the  PALACE  program,  the 
range  maps  were  generated  using  stereo  ranging,  but  the 
SLAD  algorithm  can  be  used  with  range  maps  from  any 
sensor,  including  active  as  well  as  passive  sensors. 

The  constraint  values  used  for  the  PALACE  flight 
trials  are  based  on  the  geometry  and  performance  limits 
of  the  Yamaha  RMAX.  The  landing  site  slope  constraint 
value  was  set  to  15  degrees,  although  all  landings 
performed  with  the  RMAX  were  on  slopes  below  5 
degrees.  The  safe  distance  constraint  is  set  to  7  m  to 
ensure  the  algorithm  selects  a  landing  point  with  an 
open-area  diameter  of  at  least  7  m.  This  leaves  a  margin 
of  2  m  from  the  rotor  tip  to  the  nearest  obstacle. 

The  success  or  failure  of  the  SLAD  algorithm  is 
based  its  performance  in  choosing  a  valid  landing  site.  A 
success  occurs  when  the  algorithm  correctly  identifies  a 
safe  landing  point  when  one  exists,  or  when  it  correctly 
identifies  no  safe  landing  sites  when  none  exist. 

The  performance  of  the  SLAD  algorithm  was 
evaluated  in-flight  for  a  number  of  different  landing 
scenarios,  with  variations  in  surface  texture  (asphalt, 
grass,  concrete  and  gravel),  obstacle  field  (size,  density 
and  spacing),  height  above  the  terrain,  and  SLAD 
constraint  values.  With  an  accurate  representation  of  the 
surface  terrain  in  the  form  of  a  3D  range  map,  the  SLAD 
algorithm  was  able  to  correctly  identify  which  portions 
of  the  field  of  view  were  safe  to  land,  and  which  were 
not.  For  cases  where  the  stereo  range  map  contains  more 
noise  than  expected,  the  SLAD  algorithm  is  more 
constrained  by  the  additional  surface  roughness  and 
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Figure  9.  Landing  site  selection  results  from  24  m  AGL 
over  a  gravel  surface. 


could  potentially  reject  areas  that  are  safe  for  landing. 
This  results  in  a  conservative  system  where  safe  landing 
areas  can  potentially  be  deemed  unsafe  because  of  the 
additional  noise. 

Figure  9  shows  the  SLAD  results  from  24  m  AGL 
for  one  particular  landing  to  a  gravel  surface.  The 
landing  zone  contained  a  golf  cart  (1.8  m  tall)  and  two 
boxes  (1.6  m  tall).  For  gravel,  the  stereo  ranging 
performance  is  similar  to  grass,  and  the  roughness 
constraint  is  set  at  1 .0  m  from  24  m  AGL.  For  this  case, 
the  box  near  the  center  of  the  image  is  recognized  as  an 
obstacle  and  rejected  as  a  safe  point  to  land.  The  chosen 
safe  landing  point  (shown  with  the  black  circle)  is  within 
the  safe  region  and  maximizes  the  distance  from  the  box 
and  the  edge  of  the  window. 

CONCLUSIONS 

This  paper  provided  an  overview  of  the  PALACE 
program  and  the  methods  of  integrating  the  machine 
vision  technologies  with  realistic  vehicle  dynamics  and 
control  laws  for  autonomous  landings  of  RUAVs. 
Results  from  flight  trials  to  evaluate  and  verify  the 
performance  of  the  machine-vision  components  were 
reported. 

The  results  show  that  precise  autonomous  landings 
to  unprepared  sites  are  possible  without  ground-based 
instrumentation  or  markings,  and  without  GPS.  This  has 
been  demonstrated  in-flight  with  upwards  of  40 
successful  landings  to  date  on  various  surfaces  and 
obstacle  fields. 


A  comparison  between  the  quantitative  objectives  of 
the  PALACE  program  (listed  in  Table  1)  and  the  values 
measured  in  flight  is  shown  in  Table  3.  The  vision- 
based  self-localization  system  is  able  to  meet  the 
requirements  of  drift  (landing  accuracy)  and  processing 
time  with  a  performance  that  this  robust  for  different 
surfaces,  lighting  conditions,  and  atmospheric 
conditions.  The  landing  site  selection  algorithm  was  also 
able  to  meet  the  requirements  by  rejecting  surfaces  with 
obstacles  greater  than  10  cm  at  a  success  rate  greater 
than  98%  with  a  calculation  time  of  less  than  3  seconds. 


Table  3.  PALACE  program  quantitative  metrics  and 
flight  measured  values. 


Quantitative  Metric 

Project 

Objective 

Measured 

Values 

Landing  Site  Size 

<  7.0  m 

<  7.0  m 

Landing  Surface  Slope 

<  15  deg 

<  15  deg 

Landing  Surface  Roughness 

<  10  cm 

<  10  cm 

Landing  Accuracy 

<  1 .25  m 

<  1 .00  m 

Feature-Tracking  Time 

< 100  msec 

<  60  msec 

SLAD  Calculation  Time 

<  5  sec 

<  3  sec 

SLAD  Success  Rate 

>  98% 

>  98% 
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PALACE 

Precision  Autonomous  Landing  Adaptive  Control  Experiment 


•  Motivation: 

-  Requirement  for  precision  autonomous  UAV  VTOL  landing  capability  at 
unprepared  sites  (GPS  occluded  or  denied)  in  variable  winds 


-  Enable  perch  and  stare  surveillance,  precision  UAV  supply  delivery  and 
FARP  operations,  recovery,  ground  loiter  and  forced  landing 
contingencies 


-  Potential  Applications: 

•  Fire  Scout,  MAV  (ducted  fan),  etc. 

•  Manned  platforms,  brown-out,  etc. 

*  Key  Challenges: 

-  Non-cooperative  landing  site 

-  Obstacles  in  landing  zone 

-  GPS  denied/occluded  environment 

-  On-board  intelligent  decision  making 


PALACE 

Precision  Autonomous  Landing  Adaptive  Control  Experiment 


PALACE  Approach: 

-  Machine  vision  algorithms  for  landing  site 
selection  and  self-localization  without  GPS 

-  Simulation  environment  to  integrate  and 
evaluate  real-time  vision  routines,  and 
vehicle  dynamics  and  control 

-  Advanced  control  modes  for  transitions 
between  machine  vision,  GPS,  Inertial 
Navigation,  etc.  on  autonomous  test-bed 

-  Validation  and  flight  trials  of  landing 
capabilities  on  Yamaha  RMAX 


-  PALACE  work  performed  under  a  three- 
year  Army  ATO  (IV.AV.2003.01) 

•  Concluded  end  of  FY05 


Presentation  Outline 


•  PALACE  program  overview 

•  Machine  vision  technologies 

•  PALACE  flight  trials 

•  Conclusions  /  current  activities 


Stereo  Range  Mapping 


•  Elevation  map  in  inertial  coordinates  calculated  using  stereo  images: 

-  Based  on  disparities  between  features  in  left  and  right  images 

-  Corrected  for  camera  mounting  angle  and  helicopter  attitudes 

•  Processing  time  varies  with  resolution  (approx.  1-5  seconds) 


Safe  Landing  Area  Determination  (SLAD) 


Elevation  Map  Landing  Site  Selection  Results 


•  SLAD  algorithm  finds  all  valid  landing  sites  by  applying  constraints: 

-  Landing  point  slope  must  be  below  a  maximum  (15  deg  for  this  case) 

-  Open  area  diameter  must  be  above  a  minimum  (7m  diameter  for  RMAX) 

-  Obstacle  sizes  (surface  roughness)  must  be  below  a  maximum  (based  on  altitude) 


Optimum  landing  point  best  meets  constraints  (1-2  sec.  of  processing  time) 

SLAD  algorithm  works  separately  from  the  stereo  ranging  and  can  be  used  with 
range  maps  from  any  sensor  (passive  or  active) 


Monocular  Position  Estimation  (MPE) 


•  Feature  tracking  algorithm  locks  onto  selected  landing  point 

•  Self-localization  with  respect  to  fixed  landing  point  using: 

-  Pixel  location  within  camera  image 

-  Laser  slant  range  to  center  of  camera  image 

-  Helicopter  attitudes  and  heading 

•  MPE  system  provides  position  estimates  at  10Hz  (same  rate  as  GPS) 


Presentation  Outline 


•  PALACE  program  overview 

•  Machine  vision  technologies 

•  PALACE  flight  trials 

•  Conclusions  /  current  activities 


Vibration-isolated  wjng 


1  meter  stereo  baseline 


Yamaha  RMAX 

194  lbs  GW,  65  lbs  payload 
3m  rotor  diameter 
One  hour  endurance 


PALACE  Flight  Demonstration  Vehicle 

Autonomous  Rotorcraft  Project  (ARP) 


Laser  range 
finder 


Avionics  Payload 

Crossbow  AH RS  IMU 
Ashtech  DGPS 

PCI 04+  flight  control  computer 
Compact  PCI  experimentation  computer 


PALACE  Components: 

-  IMU  data  (airframe) 

-  GPS  data  (airframe) 

-  Vision  processing  computer 

-  Two  digital  cameras  (stereo) 

-  Laser  range  finder 


Estimated  system  weight: 

-  5  pounds 


Flight  Performance  Evaluation 


PALACE  Flight  Testing  and  Demonstration: 

•  Component  flight  trials: 

-  MPE  performance  for  hover,  descent  and  landing 

-  Stereo  ranging  performance  with  different  surfaces 

-  SLAD  performance  with  different  obstacle  fields 

•  Integrated  landing  system  flight  trials  to  verify 
PALACE  system  functionality,  transitions  and 
mode  switching 


Approximately  30  flights  for 
PALACE  development, 
validation  and  evaluation 
testing 

Public  flight  demonstration: 

-  January  31, 2006 

-  ~  40  attendees  from 
industry,  government  and 
academia 


JSfiJ  J  ALT(m): 

$4%  Port  FoyrvJ 


Autonomous  Landing  Video  --  Flight 


Monocular  Position  Estimation  Flight  Evaluation 


Vision-Based  Localization  --  Key  Results: 

•  Tracking  drift  of  <  1m  in  6  minutes  of  maneuvering  without  GPS  --  Typical  result 

•  System  performance  is  robust  to  various: 

-  Surfaces  (grass,  concrete,  asphalt,  gravel) 

-  Lighting  conditions  (overcast,  sun) 

-  Atmospheric  conditions  (wind  speed,  direction,  turbulence) 

•  System  runs  at  10Hz  --  MPE  processing  time  is  45  msec 


Stereo  Ranging  Flight  Evaluation 


Obstacle  Height  Resolution 


AltiudeAGL(rr$  0  m  12  m  18  m  24  m  30  m 
CbstadeHeidjt  (m) 


Obstacle  Size 
Constraint  Values 


Stereo  Ranging  --  Key  Results: 


r 

Height  AGL  (m) 

Allowable  Obstacle  Size  (m) 

Measured  for  asphalt 

30 

1.2 

24 

0.8 

18 

0.4 

12 

0.25 

6 

L _ 

0.10 

_ A 

Stereo  performance  fairly  independent  of  surface  texture  (grass,  concrete,  asphalt,  gravel) 

-  Some  dependence  on  surface  directionality  and  lines 

Measured  obstacle  heights  were  within  about  20%  of  actual  height  for  all  cases: 

-  Results  improve  to  within  10%  below  20m  AGL 

Stereo  ranging  accuracy/resolution  improves  as  the  altitude  decreases  during  descent: 

-  Reject  obstacles  >1 0cm  from  6m  AGL 


Landing  Site  Selection  Flight  Evaluation 


Autonomous  Landing  Site  Selection  --  Key  Results: 

•  Landing  site  selection  algorithm  effectively  refines  the  location  of  the  landing  point 
and  the  helicopter  descends  and  terrain  sensing  becomes  more  accurate: 

-  Reliably  rejects  sites  with  obstacles  >  10cm  in  height 

•  Obstacle  size  constraint  values  must  be  tuned  to  produce  good  SLAD  results: 

-  Key  is  accurate  stereo  range  data  (real-time  sensing) 


Quantitative  Metrics 


Measured  performance  metrics  in  flight  were  evaluated  against  the  objectives 
of  the  PALACE  program 


Quantitative  Metric 


Project 

Objective 


Measured 

Values 


Landing  Accuracy 
Position  Estimation  Time 

<  1.25  m 
< 100  msec 

<  1.00  m 
<  50  msec 

Landing  Surface  Slope 

<15  deg 

<  15  deg 

Landing  Site  Size 

<  6.25  m 

<  6.25  m 

Allowable  Obstacle  Size 

<  10  cm 

<  10  cm 

SLAD  Calculation  Time 

<  5  sec 

<  3  sec 

SLAD  Success  Rate 

>  98% 

>  98% 

Conclusions 


•  Precise  autonomous  landings  to  unprepared  sites  are  possible  without 
ground  based  instrumentation  and  without  GPS 

•  Fully  autonomous  with  all  decision  making  and  processing  on-board 

•  No  operator  interaction  required 

•  All  objective  performance  metrics  were  met 

•  Repeatable  and  robust  with  >  40  successful  landings  to  various  surfaces 
and  obstacle  fields 

•  Concludes  3-year  successful  S&T  program  to  demonstrate  autonomous 
landing  system  feasibility  -->  Good  basis  to  move  forward  with  a  more 
directed  application 


Current  Status  and  Potential  Applications 


Current  Status: 

•  TRL  6  --  Prototype  system  demonstration  in  a  relevant  environment 

•  Higher  TRL  requires: 

-  Prototype  hardware  needs  hardening  and  qualification 

-  Additional  flying  is  required  in  less  than  ideal  conditions  --  Fog/smoke,  rain,  etc. 

•  Investigating  passive  (EO  cameras)  versus  active  (scanning  laser)  terrain  sensing  for 
autonomous  landing 

-  American  Helicopter  Society  UAV  Specialists  Meeting  paper  (Phoenix,  AZ,  January  2007) 

•  PALACE  work  is  part  of  a  larger  program  at  the  AFDD  investigating  obstacle  field 
navigation  and  UAV  control  and  flying  qualities 

Potential  Future  Applications: 

•  VTOL  UAV  autonomous  landing,  including  Firescout,  MAV  (ducted  fan),  etc. 


Possible  application  to  manned  aircraft  for:  emergency  landing,  precision  landing, 
situational  awareness  and  landing  in  degraded  visual  environments  (ie.  brown-out) 


PALACE 

Precision  Autonomous  Landing  Adaptive  Control  Experiment 


Questions  ? 


Colin  Theodore 
Mail  Stop  T12B-2 
Ames  Research  Center 
Moffett  Field,  CA,  94035 


Mark  Tischler 
Mail  Stop  T12B-2 
Ames  Research  Center 
Moffett  Field,  CA,  94035 


ctheodore@mail.arc.nasa.gov 
Phone:  650-604-1180 


mtischler@mail.arc.nasa.gov 
Phone:  650-604-5563 


